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ABSTRACT 

It  is  a  well-known  fact  that  the  major  degradation  source  on  EO  imaging  underwater  is  from  scattering  by  the 
medium  itself  and  the  constituents  within,  namely  particles  of  various  origins  and  sizes.  Recent  research  indicates 
that  under  certain  conditions,  such  degradations  could  be  caused  mainly  by  the  variations  of  index  of  refraction 
associated  with  temperature  and  salinity  micro-structures  in  the  ocean.  These  would  inherently  affect  the  optical 
signal  transmission  underwater,  which  is  of  vital  interest  to  both  civilian  and  military  applications,  as  they  could 
include  diver  visibility,  search  and  rescue,  mine  detection  and  identification,  and  optical  communication.  The 
impacts  from  the  optical  turbulence  are  yet  fully  understood,  in  part  due  to  the  challenges  associated  with 
parameterization  of  individual  factors.  This  study  presents  the  initial  attempts  in  quantify  ing  the  level  of  EO  image 
degradation  due  to  optical  turbulence  in  natural  waters,  in  terms  of  modulation  transfer  functions,  and  enhancements 
using  the  lucky  patch  approaches  derived  from  optical  flow  techniques.  Image  data  collected  from  natural 
environments  during  SOTEX  (Skaneateles  Optical  Turbulence  Exercise,  July  22-31, 2010)  using  the  Image 
Measurement  Assembly  for  Subsurface  Turbulence  (IMAST)  are  presented.  Optical  properties  of  the  water  column 
were  measured  using  WETLab’s  ac-9  and  LISST  (Laser  In  Situ  Scattering  and  Transmissiometry),  in  coordination 
with  temperature,  conductivity  and  depth.  Turbulence  conditions  were  measured  by  two  different  approaches, 
namely  a  3D  Doppler  based  velocimeter  with  Conductivity/Temperature  combo,  and  a  shear  based  Vertical 
Microstructure  Profiler. 


1.  INTRODUCTION 

The  amount  of  image  degradation  in  the  underwater  environment  is  a  function  of  optical  properties  of  the  water.  The 
cause  of  the  degradation  has  been  mostly  attributed  to  the  dirt,  or  inorganic  as  well  as  organic  particles  in  the  water 
and  it  is  rightly  so.  Most  research  has  been  focusing  on  reducing  the  impact  of  particle  scattering  by  means  of 
discriminating  scattering  photos  involving  polarization,  range  gating,  modulation,  and  by  means  of  restoration  via 
deconvolution  [1-6].  However,  in  clean  oceanic  or  lake  waters,  another  factor  could  come  into  play.  This  is  the 
scattering  by  optical  turbulence,  which  is  the  result  of  the  variations  of  the  index  of  refraction  of  the  medium.  This  is 
mostly  associated  with  the  turbulence  structures  of  the  medium,  or  water  body  in  our  case.  Degradation  of  the  image 
quality  in  a  scattering  medium  involving  turbulence  has  been  studied  mostly  in  atmosphere.  These  studies  are 
mainly  focused  on  modeling  the  optical  transfer  function,  in  an  effort  to  restore  the  images  obtained,  such  as  in  air 
reconnaissance  or  astronomy  studies  [7,  8].  Little  has  been  done  regarding  the  turbulence  effects  on  imaging 
formation  in  water,  mainly  due  to  the  dominant  particle  scattering  and  strong  attenuation  associated,  as  well  as 
challenges  in  providing  a  stable  platform  and  quantifying  the  level  of  turbulence.  It  has  been  commonly  observed 
that  visibility  can  quickly  reduce  to  zero  in  a  few  meters,  or  even  a  few  feet  in  coastal  waters,  especially  those  inside 
a  harbor,  or  estuarine  areas  like  Mississippi.  The  same  applies  to  regions  of  strong  re-suspension  from  the  bottom, 
both  in  coastal  regions  as  well  as  in  the  deep  sea.  However,  the  effects  of  turbulence  have  been  postulated  to  have 
impacts  over  long  image  transmission  ranges  [9],  which  has  been  supported  by  light  scattering  measurements  and 
simulations  [10].  Under  extreme  conditions,  observations  have  been  made  that  involve  target  ranges  within  a  few 
feet  [I  I].  The  images  obtained  under  such  conditions  are  often  severely  degraded  or  blurred,  on  par  with  or  more 
than  those  caused  by  particle  scattering.  Overcoming  such  challenges  to  increase  both  the  reach  and  the  resolution  is 
of  vital  importance  to  military  and  civilian  applications,  from  mine  detection  and  identification,  to  diver  visibility, 
undersea  communication,  and  search  and  rescue  operations.  It  is  important  to  establish  a  good  understanding  of  the 
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limiting  factors  under  different  conditions.  The  SU1M  model  [12]  was  developed  to  address  this  issue  and  it  has 
been  shown  that  on  average,  the  relative  contribution  of  different  components  [3,  13]  in  underwater  imaging 
applications  can  be  expressed  in  terms  of  the  optical  transfer  function  (OTF)  as  dot  products  of  each  component.  A 
turbulence  intensity  parameter  (S„)  is  used  to  describe  the  optical  turbulence  intensity  that  is  dependent  on  the 
structure  function,  which  can  be  further  expressed  in  terms  of  the  turbulence  dissipation  rate  of  temperature,  salinity 
and  kinetic  energy,  assuming  Kolmogorov  power  spectrum  type.  Here  we  present  the  results  of  a  recent  field 
exercise  and  efforts  associated  with  the  model  validation.  Since  the  phase  information  can  be  ignored  under 
incoherent  imaging  conditions,  the  magnitude  of  the  OTF  or  the  modulation  transfer  function  (MTF)  will  be  used 
interchangeably  here.  A  new  restoration  algorithm  was  developed  and  applied  to  the  turbulence  degraded  images  to 
compensate  for  the  impact  of  turbulence. 


2.  SOTEX  EXPERIMENT 

Skaneateles  Optical  Turbulence  Exercise  (SOTEX)  was  carried  out  from  July  22-31  2010  in  one  of  the  finger  lakes 
in  upstate  New  York,  aimed  to  examine  the  above  mentioned  impacts  in  the  field.  Optical  turbulence  underwater  is 
primarily  a  function  of  temperature  structure,  although  salinity  variations  could,  at  times,  contribute  to  strong  optical 
turbulence  [11].  Intensified  thermoclines  in  natural  environments  provide  a  convenient  setup  to  examine  this  chaotic 
process.  Details  of  SOTEX  can  be  found  in  previous  publications  [14,  15].  For  convenience,  the  key  elements  are 
briefly  outlined  here.  Data  were  obtained  during  a  multi-day  exercise  at  two  different  locations.  The  deeper  one  is 
near  the  center  of  the  lake  (42.8668°  N,  76.3920°  W)  over  a  sloping  bottom  with  an  approximate  depth  of  70  m, 
while  the  shallow  station  is  at  the  northern  end  of  the  lake  (42.9063°  N,  76.4058°  W)  over  a  flatter  bottom  with  an 
approximate  depth  of  50  m.  We’ll  be  using  data  primarily  from  the  former  (deeper)  location.  Skaneateles  was 
chosen  for  this  exercise  on  account  of  its  well-known  optically  clean  waters,  having  the  highest  clarity  of  any  of  the 
Finger  Lakes,  with  an  average  Secchi  depth  near  8  m  (Effler  et  al.  2007),  thus  allowing  for  imaging  under  varied 
turbulent  strength,  but  with  little  scattering  contribution  from  particulates.  Nonetheless,  a  particle  layer  was  present 
right  below  a  strong  turbulence  layer.  Only  images  above  this  layer  will  be  used  for  the  present  study. 


Figure  1.  Diagram  of  deployment  setup  show  ing  alternate  deployment  configurations:  (a)  Vector/CT  deployed 
vertically  on  optics  package,  and  (b)  Vector/CT  deployed  on  1MAST  both  vertically  and  horizontally.  Note,  in  both 
instances,  the  VMP  was  deployed  from  a  separate  vessel. 


In  order  to  quantify  the  level  of  image  degradation  due  to  optical  turbulence,  it  is  imperative  to  accurately  measure 
the  physical  conditions  from  which  such  impacts  arise.  For  this  purpose,  we  used  two  different  approaches  for 
turbulence  measurements,  namely  a  Vertical  Microstructure  Profiler  (VMP)  and  a  Vector  Velocimeter  combined 
with  a  Conductivity  and  Temperature  sensor  (Vector/CT)  [15].  We  inter-calibrated  the  two  systems  at  the  onset  of 
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the  exercise,  by  deploying  the  Vector/CT  from  the  optics  package,  while  the  VMP  was  deployed  from  a  separate 
vessel  nearby,  as  depicted  in  Figure  la.  While  the  Vector/CT  profile  consisted  of  pauses  at  particular  depths  for 
acquiring  a  time  series  of  velocities  that  would  be  used  for  turbulence  calculations,  the  VMP  profiled  continuously. 
For  quantifying  impacts  of  optical  turbulence  on  image  degradation,  the  Vector/CT  was  deployed  upon  a  rigid 
structure,  the  IMAST  (Image  Measurement  Assembly  for  Subsurface  Turbulence),  a  5m  long  structure  used  for 
acquiring  images  [14,  1 5].  The  IMAST  was  deployed  both  vertically  and  horizontally,  as  depicted  in  Figure  I  b, 
both  during  the  day  with  a  passive  imaging  target  and  at  night  with  an  active  target  made  of  an  iPad®,  and  also 
profiled  the  water  column  in  a  step  fashion,  pausing  at  each  depth  for  a  given  period  of  time  to  acquire  images  and 
velocity  time  series  at  a  given  depth.  The  VMP  was  deployed  from  a  separate  vessel,  and  profiled  continuously 
during  the  IMAST  deployment  and  the  two  turbulence  measurement  results  agreed  in  most  cases  [15]. 
Complementary  profiles  of  the  water  column  optical  properties  were  also  obtained  with  an  ac-9,  CTD,  and  Laser  ln- 
Situ  Scattering  Transmissometer  (LISST)  for  estimating  both  particle  size  distributions  and  the  volume  scattering 
function  of  the  water. 


Figure  2.  IMAST  night  deployment  configuration:  imaging  camera  and  housing  (left  end  of  the  frame);  active 
target  made  out  of  iPad  (right  end  of  the  frame);  details  on  other  sensors  including  Vector  refer  to  text. 


3.  DATA  PROCESSING  AND  RESULTS 

Standard  (USAF-1951)  and  custom-made  resolution  charts  are  used,  with  different  spatial  frequencies  for  quick 
examination  of  the  image  degradation.  A  pair  of  sample  images  is  shown  in  Figure  3.  They  were  taken  at  two 
different  depths,  one  at  2.8m,  which  is  essentially  free  of  optical  turbulence,  while  the  one  at  8.7m  is  strongly 
influenced  by  optical  turbulence  [14,  15].  Both  are  taken  under  the  horizontal  deployment  configuration.  One  can 
notice  that  despite  the  similarity  of  the  measured  optical  properties  in  beam  attenuation  at  532nm  (Fig.4),  the  image 
inside  the  strong  turbulence  layer  (right)  suffered  much  more  degradation,  compared  to  the  weaker  turbulence 
situation  (left).  The  turbulence  dissipation  rate  ranged  from  very'  low  values  up  to  10*6  mV1  at  the  highest  for  kinetic 
energy,  and  10^  °C2/s  for  temperature  (Fig.5).  These  two  images  are  obtained  from  the  same  path  length,  camera 
setting  and  target  intensity  level.  Thus  the  major  contributing  factor  is  the  optical  turbulence,  as  shown  in  Figure  5. 
Details  of  the  measurements  and  data  processing  can  be  found  in  the  reference  [15]. 
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Figure  3.  Sample  image  pair  obtained  by  IMAST  during  night  deployment  (horizontal)  of  July  27.  The 
corresponding  conditions  can  be  seen  in  Fig.  4.  The  left  was  taken  at  2.7m  depth,  while  the  right  was  from  8.7m, 
under  same  path,  camera  and  light  settings. 


Temperature  (C) 

Figure  4.  Optical  properties  (beam-c  at  532nm)  and  temperature  profile  measured  during  July  27  night  IMAST 
deployment. 
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Figure  5.  Profiles  of  turbulent  kinetic  energy  dissipation  rates,  e  (mV3)  determined  from  VMP  (blue  triangle)  and 
Vector  (red  open  circle)  measurements,  and  thermal  dissipation  rates,  x^CV1)  determined  from  VMP  (black  dot) 
for  July  27  day  time  deployments.  The  structures  maintained  similar  strengths  into  the  night  deployment  time  frame. 


One  can  clearly  see  from  Fig.  3  that  the  level  of  degradation  is  different  for  these  images  obtained  at  different 
depths,  which  correspond  to  different  turbulence  conditions.  To  the  first  order,  we  can  quantify  the  level  of  the 
image  degradation  in  terms  of  the  MTF,  which  describes  the  total  system  response  at  different  spatial  frequencies. 
Granted,  this  might  not  be  the  best  approach  for  turbulence  degraded  images,  especially  under  high  levels  of 
distortion.  However,  we  intend  to  estimate  the  long  exposure  (averaged)  impacts,  and  such  an  approach  is  acceptable 
for  this  first  order  calculation.  There  are  numerous  methods  to  derive  the  MTF  from  a  given  image.  We  adopted  the 
standard  slant  edge  technique  [16,  1 7],  using  different  portions  of  the  image.  The  results  are  shown  in  Figure  6  (a-d). 
These  are  obtained  from  an  active  source  (one  way  path)  during  the  nighttime  deployment  of  July  27,  to  minimize 
path  radiance.  Fig.  6a  shows  the  averaged  MTF  at  the  shallower  depth  (2.8m),  compared  to  several  cases 
(sequences,  denoted  A,  B  and  C)  at  the  deeper  depth  where  optical  turbulence  is  strong.  For  each  of  the  image 
sequences  at  8.7m,  the  MTFs  are  estimated  using  the  same  region  of  interest  (ROI)  of  consecutive  frames,  over  10- 
frames  for  long  exposure,  to  be  compared  to  single  (a,  b,  c)  and  averaged  results  at  2.8m,  as  well  as  modeled 
outcome  at  8.7m.  The  averaging  and  comparison  at  2.8m  is  used  to  examine  the  level  of  degradation  over  longer 
exposure.  The  SUIM  model  [12,  14]  is  used  to  incorporate  the  impacts  of  optical  turbulence  at  8.7m,  using  different 
turbulence  parameters.  It  is  worth  noting  that  the  signal  to  noise  ratio  (SNR)  cannot  be  improved  when  multiple 
frames  are  used,  as  each  individual  frame  would  typically  undergo  a  different  amount  of  degradation.  Therefore  the 
averaging  would  only  increase  the  SNR  towards  the  low  frequency  elements,  and  leave  behind  random  variations  at 
the  high  frequency  end.  This  is  necessary,  however,  in  order  to  contain  all  of  the  variations  caused  by  the  optical 
turbulence  [12]. 

Details  of  the  SUIM  model  and  application  can  be  found  in  previous  publications.  The  results  shown  in  Figs  6b-6d 
suggest  that  indeed  the  model  can  be  used  to  assess  the  level  of  impact  of  optical  turbulence  on  image  degradation 
underwater,  when  the  assumption  can  be  made  that  the  difference  in  MTFs  at  the  different  depths  is  primarily  the 
result  of  turbulence  structure.  By  applying  kinetic  energy  dissipation  rates  of  I0‘7~I0  9  m2s‘3,  and  temperature 
dissipation  rates  of  1 0-6—  1 0  s  °C2/s,  one  can  see  that  the  model  result  approximates  the  field  measurements 
reasonably  well.  The  differences  between  the  model  and  measurements  could  be  a  result  of  myriad  factors.  The 
primary'  reason  is  likely  the  difference  in  measurement  scales  in  the  temporal  domain.  This  is  troublesome, 
especially  when  spatial  fluctuations  of  the  turbulent  flow  cannot  be  treated  as  isotropic  and  statistically  homogenous. 
More  rigorous  post-processing  of  images  involving  multiple  ROIs,  and  direct  derivation  of  MTFs  using  the  whole 
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resolution  pattern  should  help  to  improve  the  results.  Another  possible  source  of  discrepancy  is  the  inconsistency  in 
measurement  scales  of  optical  properties  and  imaging  path,  both  temporally  and  spatially.  This  is  an  inherent  flaw  in 
this  approach  and  can  only  be  addressed  by  a  different  measurement  approach,  likely  involving  new  instrumentation 
not  readily  available  short  term.  The  turbulence  measurements  from  the  field  indicate  that  the  process  follows  that  of 
Kolmogorov  type  power  spectrum,  which  is  the  underlying  assumption  of  SU1M  and  has  been  met. 


Different  image  sequences  at  8  7m  vs  2  8m 


modeled  at  8  7m  vs  measurements  at  0  7m  (Case  A) 
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(b) 
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modeled  at  8  7m  vs  measurements  at  8  7m  (Case  C) 
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modeled  at  8  7m  vs  measurements  at  8.7m  (Case  B) 
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Figure  6.  Modulation  transfer  functions  (MTFs)  estimated  and  modeled  at  different  depth  based  on  same  ROI  over 
short  and  long  exposures  for  three  different  sequences  at  8.7m,  during  July  27  night  deployment. 


The  ultimate  goal  of  understanding  the  level  of  impact  of  optical  turbulence  as  well  as  particle  scattering  on  image 
degradation  is  to  use  the  knowledge  to  quantitatively  predict  the  degradation,  but  more  importantly,  to  restore  and 
enhance  the  imaging  outcome.  Due  to  the  level  of  degradation  and  distortion  involved,  new  algorithms  are  being 
developed  and  details  can  be  found  in  a  companion  paper  in  this  volume  [18].  Briefly,  this  approach  represents 
synthesis  of  “lucky-region”  fusion  and  optical  flow  based  image  warping.  The  newly  developed  multi-frame  image 
restoration  algorithm  is  applied  to  the  sets  of  images  collected  during  SOTEX,  as  shown  in  Fig.  3b.  The  sample 
results  are  shown  in  Figure  7.  One  can  notice  the  enhancement  over  long  exposure  (7b),  especially  that  around  the 
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edges,  and  high  spatial  frequency  patterns.  Deconvolution  with  the  help  of  known  particle  scattering  functions  is 
under  investigation,  which  should  further  augment  the  level  of  details  restored. 


(a)  (b)  <0 


Figure  7.  Sample  image  from  July  27  night  deployment  (left,  a),  its  long  exposure  sequence  (middle,  b)  and  the 
restored  version  (right,  c)  using  optical  flow  and  lucky  patch  fusion  technique. 


4.  SUMMARY 

To  assess  the  level  of  image  degradation  in  natural  environments  by  optical  turbulence,  a  rigid  imaging  structure, 
1MAST,  has  been  built  and  deployed  by  researchers  at  the  U.S.  Naval  Research  Lab  during  SOTEX  in  July  of  2010. 
Measurement  results  from  the  imaging  system  and  turbulence  probes  confirmed  the  previously  developed  SUIM 
model  as  a  valid  tool  in  estimation  of  image  degradation,  based  on  the  Kolmogorov-type  power  spectrum 
assumption.  Initial  attempt  of  restoration  based  on  the  optical  flow  approach  fused  with  the  lucky  patch  method 
shows  promising  results.  Improvements  on  turbulence  measurements  and  restoration  efforts  are  under  investigation. 
New  field  efforts  have  been  completed  recently  and  more  supporting  evidence  can  be  shown  in  the  near  future. 
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